Skip to content

added first Independent Q Learning experiment #922

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

added first Independent Q Learning experiment #922

wants to merge 6 commits into from

Conversation

@Mytolo
Copy link
Contributor

@Mytolo Mytolo commented Jul 7, 2023

PR Checklist

  • Update NEWS.md?
  • Unit tests for all structs / functions?
  • Integration and correctness tests using a simple env?
  • PR Review?
  • Add or update documentation?
  • Write docstrings for new methods?
    Here is an experiment for RLExperiments showing how to set up a simple independent learning with two players. (In the example, the roles of randomly choosing an action and taking create_policy -> DQN can be changed and they can also both try to approximate their Q values for action selection)

@codecov
Copy link

codecov bot commented Jul 7, 2023 •
edited
Loading

Codecov Report

Attention: Patch coverage is 0% with 12 lines in your changes missing coverage. Please review.

Project coverage is 0.00%. Comparing base (8f5ea30) to head (e5d3e6e).
Report is 60 commits behind head on main.

Files with missing lines Patch % Lines
...eps/experiments/experiments/MARL/IDQN_TicTacToe.jl 0.00% 9 Missing ⚠️
...xperiments/src/ReinforcementLearningExperiments.jl 0.00% 2 Missing ⚠️
...deps/experiments/experiments/MARL/pettingzoo_ex.jl 0.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff            @@
##            main    #922      +/-   ##
========================================
- Coverage   0.02%   0.00%   -0.03%     
========================================
  Files        215     216       +1     
  Lines       7622    7631       +9     
========================================
- Hits           2       0       -2     
- Misses      7620    7631      +11
Files with missing lines Coverage Δ
...deps/experiments/experiments/MARL/pettingzoo_ex.jl 0.00% <0.00%> (ø)
...xperiments/src/ReinforcementLearningExperiments.jl 0.00% <0.00%> (-100.00%) ⬇️
...eps/experiments/experiments/MARL/IDQN_TicTacToe.jl 0.00% <0.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Mytolo
Copy link
Contributor Author

Mytolo commented Jul 10, 2023

@HenriDeh, @jeremiahpslewis would you have a look? It is supposed to just be an example on how to use multiAgentPolicy.

HenriDeh
HenriDeh requested changes Jul 11, 2023
View reviewed changes
Copy link
Member

@HenriDeh HenriDeh left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see a problem with the implementation but it needs to be added to the tests of RLExperiments like the other ones are. Otherwise we may not see that a change breaks this example and it will live there broken until someone notices.

@Mytolo
Copy link
Contributor Author

Mytolo commented Jul 17, 2023

I don't see a problem with the implementation but it needs to be added to the tests of RLExperiments like the other ones are. Otherwise we may not see that a change breaks this example and it will live there broken until someone notices.

I this what you intended to say here @HenriDeh ? Commit 82f77f4

Panajiotis Keßler and others added 2 commits July 19, 2023 16:26
@Mytolo Mytolo force-pushed the add-TicTacToeIndependentDQN branch from a7eedd7 to d660085 Compare July 19, 2023 14:27
@Mytolo
Copy link
Contributor Author

Mytolo commented Jul 19, 2023 •
edited
Loading

FYI: rebased commit structure as it was messed up.

@Mytolo Mytolo requested a review from HenriDeh July 19, 2023 14:28
@Mytolo
added experiments to test
a2fed82 
adjusted tests and merged main
HenriDeh
HenriDeh reviewed Jul 20, 2023
View reviewed changes
Copy link
Member

@HenriDeh HenriDeh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes thank you. Gotta figure out why the CI fails now.

Mytolo
Mytolo commented Jul 20, 2023
View reviewed changes
Mytolo
Mytolo commented Jul 20, 2023
View reviewed changes
@Mytolo
added experiments to test
358ed8b 
adjusted tests and merged main
@Mytolo
Copy link
Contributor Author

Mytolo commented Jul 20, 2023

Finally, the pipeline should be correct. I needed to add some parts, but this is also helpful for other 3rd party environments once they need to be tested.

@Mytolo Mytolo requested a review from HenriDeh July 20, 2023 11:54
@Mytolo
Copy link
Contributor Author

Mytolo commented Jul 25, 2023

I don't see a problem with the implementation but it needs to be added to the tests of RLExperiments like the other ones are. Otherwise we may not see that a change breaks this example and it will live there broken until someone notices.

this should be solved now @HenriDeh ?

@HenriDeh
Copy link
Member

HenriDeh commented Aug 7, 2023

Hi @Mytolo, sorry for the delay, I was on vacations. You have conflicts to resolve then I can approve the PR for merging :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@HenriDeh HenriDeh Awaiting requested review from HenriDeh

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Mytolo @HenriDeh